Network security is currently enhanced in part by systems such as a network Intrusion Detection System (IDS), network Intrusion Prevention System (IPS), or firewall. However, many of these systems are deployed in high-bandwidth environments in which it is infeasible for the systems to monitor fully all of the network traffic received on a high-bandwidth link. Specifically, hardware limitations prevent these systems from monitoring all of the bytes in passing network traffic.
To address this problem, current systems typically rely on “traffic discard,” in which data is partially decoded and evaluated for “match potential.” It is then either scanned against known signatures or simply discarded without being scanned. In using traffic discard, current systems reduce the amount of data that they actually scan into a range that matches their processing capability. Discarded data often includes HTTP and SMTP content, which collectively make up a majority of the data crossing many typical high-bandwidth network environments. By not scanning HTTP and SMTP content, the load on these systems is significantly reduced. However, a significant risk is also created because any threat located in the content portion of the transaction (e.g., the HTTP response body instead of the HTTP response header) will be ignored by the threat detection system. The existing divide between bandwidth and computational power continues to expand, and will make the problem increasingly severe over time.
What is needed is a means for providing at least a degree of security with respect to portions of network traffic that would otherwise be excluded from analysis, e.g., due to hardware or other limitations.